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ABSTRACT 

This paper provides an empirical analysis of the impact and effectiveness of developmental 
relationships provided through academic intervention programs at a medium-size master’s level 
public university in the Northeastern United States. The programs ’ curriculum follows the Model 
of Strategic Learning’s four pillars of learning and is administered to students with diverse 
interventional needs. 

This paper presents a brief review of the literature about effective developmental relationship 
programs (mentoring and coaching) in higher education. Then, Ordinary Least Squares 
regressions, as well as paired samples t-tests, are used to test the impact of programs offered 
through developmental relationships to students with varying academic deficiencies. The 
immediate, as well as longer-term, impact and sustainability of students ’ enhanced performance is 
statistically examined. The paper concludes that students who fully take advantage of 
developmental relationships benefit the most and sustain their higher level of performance beyond 
the immediate post one-time intervention period. However, in the absence of additional 
intervention, the academic performance gains seem to subside and flatten out. 
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INTRODUCTION 

his paper provides an empirical analysis of the impact and effectiveness of academic intervention 
( y / programs provided through robust developmental relationships (peer-mentoring and academic 
coaching) to students with academic deficiencies in a medium sized university located in the 
Northeastern United States. A paired samples t-test, as well as an Ordinary Least Squares (OLS) regression 
analysis, is done to measure the immediate and longer term impact of academic intervention programs on students’ 
Grade Point Average (GPA). A brief review of relevant literature is provided next, followed by sections describing 
the structure of the academic intervention programs provided under different developmental relationships, the 
methodology, empirical results, a presentation of data, conclusions, and suggestions for further studies. 


REVIEW OF THE LITERATURE 


Developmental relationships (mentoring, academic coaching, etc.) in a dyad or group setting are a 
significant component of any effective intervention to improve academic performance of at-risk college students. Li 
and Julian (2012) argue that developmental relationships constitute the active ingredient of any effective 
intervention. They suggest that other inactive ingredients of an intervention program, such as incentive, 
accountability, and curricula, do not yield positive or lasting returns in the absence of effective developmental 
relationships. Effective developmental relationships include attachment, reciprocity, progressive complexity, and 
balance of power. Effective interventions establish a strong and reciprocal emotional attachment between the 
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mentor and the mentee over a period of time when they engage in activities with growing complexity and eventually 
with a transfer of power from the mentor to the mentee. Any academic mentoring or other interventions must invest 
in a structure with a strong developmental relationships component. 

Jacobi (1991) provides a critical review of the literature on mentoring from the mid-1970s to 1990. She 
identifies the absence of a commonly accepted definition of mentoring and the lack of a theoretical foundation since 
how mentoring helps students are two major concerns about the research done to 1991. She asserts that most 
research assumes that mentoring causes academic success without a clear demonstration of that. Many researchers 
(Campbell & Campbell, 2007; Flumerfeldt et al., 2007; Lennox-Terrion, 2010; McLean, 2004; Snowden & Hardy, 
2012) have demonstrated that mentoring has a positive impact on the undergraduate academic experience. Snowden 
and Hardly (2012) use a case study method and demonstrate how peer mentoring reduces stress and anxiety as well 
as enhances students’ participation and adds value to students’ performance. The causality between mentorship and 
students’ academic performance has been examined by many researchers (Campbell & Campbell, 2000; Ferrari, 
2004; Rhodes, 2008). For example, Rhodes (2008) uses a paired samples t-test and a Chi-square test to demonstrate 
that mentored undergraduate students had a higher GPA than non-mentored students. He also concludes that there is 
statistically significant evidence that mentored students have a higher graduation rate than non-mentored students. 
Other researchers (Jacobi, 1991; Paglis et al., 2006) have found no evidence of positive impact of mentorship on 
students’ academic performance. 

This paper uses Ordinary Least Squares (OLS) and paired samples t-tests to examine the immediate and 
longer term impact of peer-mentoring on undergraduate students’ academic performance measured by cohort GPAs 
at a large Northeastern university in the United States of America. 

STRUCTURE OF THE PROGRAM 

In this university, all undergraduate students whose GPA falls below 2.00 (on a maximum 4.00 scale) and 
business students with a GPA below 2.5 are placed on probation and offered an academic coaching program (ACP, 
at the university level) or peer-mentoring program (Academic Ally program, “AAP”) in the School of Business. 
Coaches for the ACP are graduate students who are extensively trained to provide mentoring to undergraduate 
students. The Academic Ally program trains upper division undergraduate business students during a one-week 
“boot camp” to mentor under-performing business students. Mentors and mentees meet for at least 14 group 
sessions over the course of one semester and participate in a formal and well-structured curriculum. The curriculum 
design is based on the four pillars of learning in the Model of Strategic Learning (Weinstein & Mayer, 1986). These 
are the broad components which, through their interaction, describe how students become more strategic learners. 
These components are skill, will, self-regulation and the academic environment. The potency of this model lies in 
the interaction between these components. Although the effects of using the model cannot be traced to the direct 
effect of a single element, it is useful to describe each component in this interactive model. 

The skill component refers to the types of knowledge, strategies, and tactical skills a student must possess 
to succeed. This component can be broken down to five elements. First, a student must have self-knowledge as a 
learner. Self-knowledge pertains to understanding strengths and weaknesses as a learner, personal motivation, 
attitudes and anxiety levels toward learning. Second, a student must have knowledge of what is required to 
successfully complete a given academic task so that the student knows what to think about and do in order to 
succeed. Third, a student must have a learning strategy in order to acquire, integrate and apply new learning. 
Fourth, a student must understand that new learning is often based on prior knowledge and therefore have an 
awareness of their personal knowledge base. Finally, a student must be able to place the new learning in a larger 
context, so the student will understand how new learning will potentially help them meet larger goals, whether 
personal, social, academic or occupational in nature. 

The second component of the model is will, which pertains to a student’s motivation. Motivation is 
reflected in a student’s ability to create learning goals and relate the immediate task to those goals. Motivation is 
also related to a student’s belief in their own ability to accomplish a task. Whether a student tries to accomplish 
something is often related to their underlying belief that they can, in fact, do it. 
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Self-regulation is the third component of the model. This relates to the student’s ability to utilize time 
management and adapt a systematic approach to learning. Time management refers to a student’s ability to balance 
their time among many competing demands in a way that allows them to reach their learning goals. A systematic 
approach to learning occurs when the student is able to set a goal, develop a plan, chose a strategy to implement the 
plan, monitor the implementation, modify as needed, and evaluate the results. To do so, students need to monitor 
their stress level, motivation, concentration and comprehension. 

The fourth component of the model is the academic environment, which is external to the student. This 
environment consists of factors such as available learning material and resources, teacher expectations, the nature of 
the learning activity and time constraints. The student should be able to adapt to the various types of academic 
environments in which they must perform in order to succeed. 

The program starts with simple group activities and ends with more complex tasks. This would develop a 
determined and independent mindset among mentees to become academically successful. Mentors and mentees are 
encouraged to engage in dyad exchanges outside the group meetings. 

The AAP has been offered for only two semesters. In the first pilot offering, mentees and mentors were 
matched on a voluntary basis after they had a chance to meet and become familiar with each other’s backgrounds. 
In the second semester, multiple sections of a mentoring course were offered and mentors, as well as mentees, 
registered for a convenient section without any prior knowledge about each other. In this setting, mentor-mentee 
matching was basically random and it seemed that interactions between mentees and mentors outside the class 
sessions were very limited compared to the first semester cohort. The peer-mentoring programs are intended to 
capitalize on a process (developmental relationships) that provides comfort and trust to learn from a peer. The major 
mentoring functions include advice and guidance, sponsorship and advocacy, and training and instructions. These 
are significant mentoring functions that most other researchers have identified in a typical mentorship (Jacobi, 
1991). 

METHODOLOGY 

Jacobi (1991) asserts that many empirical studies on mentoring and academic performance assume that 
there is causality between the two based on measured correlations. In addition, indirect measures (such as surveys) 
of positive impact of mentorship on students’ academic performance based on case studies, as well as small sample 
size, limit external validity and internal consistency of empirical results. This paper attempts to overcome most of 
these flaws by using paired samples t-tests, as well as a stepwise Ordinary Least Squares, to establish causality in 
addition to measure correlations. 

Participants 

Two-hundred twenty-five undergraduate students with a GPA below 2.00 were required to participate in 
the ACP. Only 201 of them completed the program successfully and 24 failed to do so. These students belonged to 
four cohorts who were offered mentorship from Fall 2011 to Spring 2013. At the School of Business, two cohorts, 
composed of 32 students with a GPA between 2.00 and 2.50, participated in the School’s AAP. Twenty-seven of 
them completed the AAP successfully and five failed to complete the program. Additionally, a control group was 
created by random selection from all School of Business students who were invited to participate in AAP, but 
declined, and for whom GPA data was available for Spring 2012, Fall 2012 and Spring 2013 semesters (23 from the 
Spring 2012 warning list and 21 from Fall 2012 warning list). 

The overall average GPA of each cohort at the end of semester before mentorship (GPA-B) and four 
semesters after completion of mentorship (GPA-A1 through A4) were used as a measure of academic performance. 
For the ACP, a one-way ANOVA test was performed on all cohorts to ensure that they could be treated as a single 
sample. For the AAP, an independent samples t-test was performed on both cohorts to ensure that they could be 
treated as a single sample. In both cases, no significant differences between cohorts were found at baseline for 
either the ACP or the AAP. 
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Paired Samples T-test Estimation 

Table 1 presents the estimation results of paired samples t-tests for the Academic Coaching and Academic 
Ally Programs for students who successfully completed the peer-mentoring programs and those who did not. A 
series of paired samples t-tests were performed for students who successfully completed the ACP (n = 201). The 
cohort’s average GPA one semester after completion of the mentorship (GPA-A1) significantly increased (0.83 on a 
scale of 1 to 4) compared to before mentorship. Conversely, no significant difference was found between average 
GPA before and after mentorship for 24 students who did not successfully complete the program. The positive 
effect of participating in and successfully completing the ACP was maintained for two and three semesters after 
successful completion of the program (see B/A2, and B/A3 in Table 1), but due to the small sample size, no 
significant result was found at a one or five percent significance levels. 

Also a series of paired samples /-tests similar to those done for the ACP were performed for students who 
successfully completed the AAP in the School of business (n = 27). Post-mentorship (B/Al, Table 1) average GPA 
for students who successfully completed the AA program improved by 0.37 on a scale of 1 to 4. However, and as 
expected, no significant difference was found between pre and post-mentorship GPA of the 49 students who either 
participated but did not complete the program or did not participate. The average GPA for the group who completed 
the AA program improved by 0.38 two semesters after completion of the program (See B/A2, Table 1); however, 
due to the small sample size (n = 12), the results are not statistically significant. 


Table 1: Paired Samples t-Test Comparisons between Semester 
CPA’s for Academic Coaching and Academic Ally Programs 



Successfully Completed 

Not Successfully Completed 

Pair 

n 

MD 

SD 

t 

df 

P 

n 

MD 

SD 

t 

df 

P 

Academic Coaching Program 

B/Al 

201 

0.83 

0.89 

13.27 

200 

<.001 

24 

0.02 

1.21 

-.062 

23 

.951 

B/A2 

100 

0.87 

0.85 

10.22 

99 

<.001 







B/A3 

61 

0.76 

0.85 

7.01 

60 

<.001 







B/A4 

14 

0.61 

1.28 

1.79 

13 

.097 







Academic Ally Program 

B/Al 

27 

0.37 

0.59 

3.28 

26 

.003 

49 

-.10 

1.05 

0.69 

48 

.494 

B/A2 

12 

0.38 

0.90 

1.48 

11 

.167 








Ordinary Least-Squares Estimation 

To further explore the causal effect of successfully completing the Academic Coaching and Academic Ally 
programs on post-intervention GPA, a stepwise OLS estimation was used to regress post-mentorship average GPA 
(A1 to A4) on a constant, the pre-mentorship GPA, and a dummy variable where D = 1 for successful completion of 
the programs and D = 0 for non-completion of the program. The estimation results are presented in Tables 2 and 3 
for the ACP and AAP, respectively. 

The dummy variable that represents the impact of successfully completing peer-mentoring program (ACP) 
is a significant predictor of up to three semester post-mentorship GPA (A1 to A3). Pre-mentorship GPA is also a 
significant predictor of one-semester post-intervention GPA and approaches significance for two semesters post¬ 
intervention GPA. The R-squares indicate that, at most, 21 percent of variations in post-successful completion of 
the mentoring program GPA are explained by pre-mentorship GPA and successful completion of the mentoring 
program. However, the initial impact subsides to only three percent after two semesters. 
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Table 2: Summary of Stepwise Ordinary Least Squares Regression Analyses for Variables Predicting Post-Intervention GPAs for the Academic Coaching Program 


Explanatory 

Variable 

Model 1 

Model 2 

Coefficient 

St. Error 

Standardized 

Coefficient 

t 

Coefficient 

St. Error 

Standardized 

Coefficient 

t 

Dependent Variable: First Semester After Intervention (Al) 

Constant 

1.023 

.188 


5.450*** 

.533 

.208 


2.562* 

Dummy (Successfully 
Completed =1) 

1.180 

.199 

.370 

5.941*** 

1.024 

.193 

.321 

5.312*** 

Pre-Intervention GPA 





.472 

.101 

.283 

4.682*** 

R 1 


.137 




.214 



F 


35.293*** 




30.263*** 



Dependent Variable: S 

econd Semester After Intervention (A2) 

Constant 

1.332 

.361 


3.691*** 

.963 

.406 


2.373** 

Dummy (Successfully 
Completed = 1) 

.985 

.370 

.254 

2.665** 

.947 

.366 

.244 

2.591* 

Pre-Intervention GPA 





.281 

.147 

.180 

1.906 

R 1 


.065 




.097 



F 


7.103** 




5.459** 



Dependent Variable: Third Semester After Intervention 

(A3) 

Constant 

2.082 

.299 


5.220*** 

1.661 

.502 


3.307** 

Dummy (Successfully 
Completed =1) 

.193 

.412 

.059 

0.469 

.112 

.413 

.034 

0.271 

Pre-Intervention GPA 





.332 

.243 

.173 

1.367 

R 2 


.003 




.033 



F 


.220 




1.046 



Dependent Variable: Fourth Semester After Intervention (A4) 1 

Constant 

1.926 

.768 


2.508** 





Dummy (Successfully 
Completed = 1) 









Pre-Intervention GPA 

.168 

.444 

.108 

.378 





R 1 


.012 







F 


.143 








* p < .05. ** p < .01. *** p < .001. ’All participants who completed a fourth semester after intervention (A4) also successfully completed the program. The variable “Successfully Completed” was not 
included in the model for this semester. 
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Further, the OLS estimates indicate that successfully completing the Academic Ally program significantly 
contributes to improvement of one-semester post-mentoring GPA. Also, at most 8.2 percent of variations in the 
dependent variable are explained by successfully completing the AAP. Unlike the results of the ACP, the Pre¬ 
mentorship GPA does not explain improvement in post-mentoring GPA. 


Table 3: Summary of Stepwise Ordinary Least Squares Regression Analyses for Variables 
_ Predicting Post-Intervention GPAs for the Academic Ally Program _ 


Explanatory 

Variable 

Model 1 

Model 2 

Coefficient 

St. 

Error 

Standardized 

Coefficient 

t 

Coefficient 

St. 

Error 

Standardized 

Coefficient 

t 

Dependent Variable: First Semester After Intervention (A1 


Constant 

2.326 

.105 


22.068*** 

2.091 

.329 


6.352*** 

Dummy 
(Successfully 
Completed = 1) 

.456 

.177 

.287 

2.579* 

.436 

.179 

.275 

2.433* 

Pre-Intervention 

GPA 





.106 

.140 

.085 

0.752 

R- 


.082 




.090 



F 


6.651* 




3.589* 



Dependent Variable: Second Semester After Intervention (A2) 

Constant 

2.348 

.150 


15.694*** 

2.472 

.500 


4.948*** 

Dummy 
(Successfully 
Completed = 1) 

.467 

.259 

.296 

1.804 

.483 

.270 

.306 

1.792 

Pre-Intervention 

GPA 





-.057 

.221 

-.044 

-.259 

TC 


.087 




.089 



F 


3.254 




1.616 




* p < .05. **p<.01. ***p<.001. 


Figure 1 presents Mean GPA for participants in the ACP and AAP at the end of semesters before and after 
mentorship programs. The results indicate that the positive impact of the ACP is sustained at least three semesters 
after successful completion of the program. Because only one of the AAP cohorts had completed a post mentorship 
semester at the time of this writing, complete data for the AAP was not available to fully test sustainability of the 
impact of the mentorship program. 



Figure 1: Pre and Post-Mentorship Mean Semester GPAs for Participants in the Academic Coaching (AC) 
and Academic Allies (AA) Programs (Vertical Dotted Line Represents the Intervention Semester) 
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CONCLUSIONS 

The paired samples t-tests indicate that mentoring programs at the university and school levels significantly 
contribute to an increase in the post-mentoring GPA of cohorts who successfully complete the programs. There is 
no significant difference between pre and post-mentoring GPA of cohorts that didn’t successfully complete the 
programs or did not participate. The OLS estimates clearly support a positive causal relationship between mentoring 
and academic performance improvement. Also, the significant increase in post-mentoring GPA of those in the ACP 
is sustained at least three semesters after the mentorship program is successfully completed without any additional 
interventions. To ensure external validity of the empirical results, it is suggested that at least several universities 
offer the same mentoring programs and create a much larger sample for estimation. In addition, survey data needs 
to be used to identify which components of the programs assisted the mentees to improve their academic 
performance. Also, the peer-mentor programs should be extended to include booster workshops beyond the first 
semester. This should assist to further evaluate the longer-term impact and sustainability of higher academic 
performance. 
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